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(57) Abstract: A method for deteimining a frequency of motion of an 
object from a time sequence of images of said object, said method com- 
prising the steps of applying an autocorrelation function to individual 
pixels in across said image sequence; computing an average autocorrela- 
tion for each image to produce an amplitude pattern of said average au- 
tocorrelation; applying a filter to said amplitude pattern; applying a peak 
detector to said filtered amplitude; and comparing said peak to its time 
of occunence in said image sequence to thereby determine frequency of 
motion of said object. 
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SYSTE1SJ.ANP METHOD FOR PRESSING AN MAGE SEQUENCE 

This present invention relates to the field of imaging and in particular to a system and 
method which utilizes medical imaging for detecting, monitoring and measuring the 
movement of a body or portions of the body such as the heart and lungs. 

BACKGROUND OF THE INVENTION 

Various methods have been implemented in the art for detecting and monitoring the 
periodic movement of organs within a patient's body. For example, the ubiquitous ' 
stethoscope is used for detecting the audible acoustic impulses generated by a beating heart 
The acoustic impulses are tallied in a predetermined time period, the tally corresponding to 
an average pulse rate that in turn maybe correlated to the heart beat frequency of the patient. 
Other methods make use of more advance technology for monitoring movement of the heart 
or lung. These systems may be divided into several generally categories namely: 
stethoscopes and acoustic monitors; ultrasound monitors and magnetic resonance systems; 
optical imaging technology and non-acoustic pulse echo radar monitors. By way of 
background the reader is referred to United States Patent No. 5,766,208 which provides a 
comprehensive discussion of each of the above types of systems. 

While the above methods and apparatus is suitable in certain applications for 
obtaining cardiac frequency and breathing rate, the use of such technology may not be 
feasible or efficient in other applications. In MRI imaging or X-ray angiography, the use of 
the above methods may introduce unnecessary features in the image sequence such as; for 
example, an ECG electrode may in fact obscure critical areas of a patient's pathology being 
investigated. 

On the other hand, certain cardiac imaging modalities such as X-ray angiography and 
echocardiography create dynamic image sequences of the heart or lung. For certain 
applications, it is useful or necessary to have information about the cardiac frequency in these 
image sequences. Traditional methods as described above are normally used to obtain 
cardiac frequency during an imaging apphcation. 

It has also been recognized that in certain imaging applications, such as magnetic 
resonance imaging (MRI), the measurement of MR signals exhibit smaller errors if the 
measurement of the amount of signals is synchronized to a cyclic movement of the body. 
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One such method described in United States Pallnt No. 5,987,348 that describes an ECG- 
triggered magnetic resonance imaging method and apparatus. The patent discloses a method 
of imaging a portion of a body placed in a static magnetic field by means for magnetic 
resonance and determines an acquisition period from ECG data in order to synchronize 
measurement of MR signals to a cyclic movement of the body. A drawback of this patent is 
that it requires a separate ECG measurement instrument The acquisition sensors of which 
must be carefully placed so as not to obstruct the MR imaging at the location of interest. 

Accordingly, there is a need for a method and system for determining these periodic 
movements of the patient's organs, such as cardiac frequency, directly from the image 
sequence without the need for external equipment such as an ECG and such like and which 
mitigates at least some of the above disadvantages. 

SUMMARY OF THE INVENTION 

The present invention seeks to provide a solution to the problem of detennining 
periodic movement of selected portions of a body directly from an image sequence without 
the need for external measuring equipment such as ECGs. 

In accordance with this invention, there is provided a method for measuring a periodicity 
of a recurring pattern contained in a time sequence of images comprising the steps of: 

(a) processing a plurality of said images for detecting a pattern according to a first 
criterion; and 

(b) processing the detected pattern according to a second criterion to determine a 
measure of periodicity of said recurring pattern. 

In accordance with a further aspect the first criterion is an autocorrelation function. 
In accordance with a further aspect the first criterion is a Fourier analysis. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features of the preferred embodiments of the invention will become 
more apparent in the following detailed description in which reference is made to the 
appended drawings wherein: 

Figure 1 is a schematic diagram depicting the components of a frequency monitoring 
and detecting system according to an embodiment of the present invention; 

Figure 2 is a schematic diagram showing a sequence of images; 
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J^?.3.is a schematic diagram showing ^arrangements pixels in a single image; 
Figure 4 is a generalized flow chart showing the determination of a heartbeat 
frequency according to an embodiment of the present invention; 

Figure 5 is a graphical plot showing an autocorrelation coefficient versus image 
number; 

Figure 6 is a section of the graphical plot of fr gure 5; and 
Figure 7 is a graphical plot showing an output of a highpass filter applied to the 
graphical plot of figure 5. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following description, like numerals refer to like structures in the drawings. 
Referring to figure 1, a system for detecting, monitoring and displaying periodic motion of a 
subject or portions of a subject, from a sequence of images, in accordance with one 
embodiment of the present invention is indicated generally at numeral 10. The present 
embodiment will be described with reference to a specific cardiac imaging modality such as 
x-ray angiography. Other imaging modalities may equally well be used to create the dynamic 
image sequences. The embodiment is also described with reference to the determination of a 
heartbeat frequency. The invention may be applied to detecting motion in the lungs, breathing 
or one or more combinations of motions present in a sequence of images. 

Turning back to figure 1, the system 10 comprises an x-ray source 12, a timebase 14 
for triggering the x-ray source 12 at a predetermined rate, an image plane 1 6 for receiving 
two dimensional x-ray image of the irradiated subject, a computer 20, a program 30 for 
running on me computer 20, a database 22, user interface 24, and a display 26. Usually, 
during an imaging procedure, x-rays are passed through the patient 1 8 and an image is 
projected onto the plane 16. The image is converted to a two dimensional array of pixels and 
forwarded to the computer 20 for viewing as a displayed image on display 26 or stored as a 
sequence of images 28 in a data-set in the database 22. The program 30 may process the 
data-set 28 or process the images in real time to determine motion of the portion of the body 
being irradiated. The user input 24 provides facility for an operator to interact with the 
system, and more particularly, for selecting areas of the displayed image 26 for isolating 
areas of the image to be processed or to set various parameters of the system. The methods 
and systems for acquiring, displaying and generating image sequences is well known in the 
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ffr is described in various patents and patent appticatiorf^owned by the current assignee and 
incorporated herein by reference. 

The computer 20 is programmed in a manner, which is described in more detail 
below, to operate in accordance with the following description of the preferred embodiment. 

As shown in figure 3(a) and (b), typically each image is comprised of a two 
dimensional array of individual pixels 304, 

The present invention is based on the observation that motion may be reflected in 
successive sequences of images. The individual pixels 304 across the sequence of images 
may show variation in intensity in synchronization with the motion being monitored. For 
example, in order to detect cardiac motion, images will be taken of the heart and its 
surrounding areas. Thus, image areas, which are covered by the heart, will show changes in 
brightness in synchronism with the heartbeat It may be noted that some pixels in the areas 
covered by the heart may not show any variation in a sequence of images, such as a pixel, 
which is contained within the area of heart during acquisition of the image sequence. On the 
other hand pixels in regions immediately outside of the heart may not show any variation, as 
these regions may also not move. Finally, all pixels may show variation synchronized with 
breathing. However breathing motion has in general a different frequency than the heartbeat. 
Thus if the heartbeat frequency is to be monitored some filtering may be employed to select 
the motion or frequency of the cycle of interest. 

Turning now to figure 4, there is shown a flow chart that depicts the operation of the 
program 30 for controlling the computer 20. The sequence of steps performed by the program 
is indicated by title sequence of boxes 402 to 414. In box 402, the program inputs a sequence 
of images 28 for processing along with a value indicative of the image generation frequency 
f. Depending on the imaging technique and the frequency of the cycle being measured, a new 
image should be generated several dozen times per cycle or at a rate f. As seen in figure 3(b) 
the rows are represented along an X-axis and the columns of the image are represented along 
a Y-axis. Each image for different times is referenced through a time parameter t and the 
value p(x,y,t) corresponds to the intensity of an pixel at location x,y for the image at time t 

At block 404, the program computes a periodicity measurement for the sequence of 
images for each frequency of the cycle of interest. The expectation would be that for the 
frequency of the heartbeat, the periodicity measure would be strong, and that it would also be 
strong for breathing motion (if present). Various mathematical techniques can be used, 
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^Ipo 1 ? ?ourier analysis, and j^opocrelatioii in time for egracting this periodicicty. The - 
invention simply relies on some periodicity measure, regardless of how it has been obtained 
In a preferred embodiment the autocorrelation is computed over time t for each pixel position 
x,y. Those pixels whose intensities change periodically will have large autocorrelations at 
the frequencies corresponding to the period motion. Other pixels may have low 
autocorrelations. Autocorrelation coefficients measure the degree of correlation between 
neighboring data observations in a time series. Assuming the time series is y t = p(x,y,t) t=l,2, 
.. ., the autocorrelation coefficient is estimated from sampling observations as follows: 

n-k _ 

r ^ _ t*i 

V <=i V /=i 

where r k describes the autocorrelation of pixels andj^ . These coefficients r k are stored in 
memory. 

At block 406, the program computes the average autocorrelation for all pixel positions 
x,y for each image in the sequence. The average autocorrelation number 7 t ,where t = 1,2,3 .., 
corresponding to image t = 1,2,3 .., is stored in memory for further processing 
At block 408, the program computes those frequencies, which are within the expected range 

by applying a high-pass filter over the average autocorrelation r ' . The high-pass filter over 
the autocorrelation will retain those frequencies which are within the expected range of me 
periodic motion of interest, but eliminate frequencies that are lower, such as breathing. In 
practical applications, heartrates, for example that are outside of a well-defined range will 
lead to an interruption of the clinical procedure. 

The filter comer frequency is chosen to be somewhere between the two frequencies at 
issue, heart beat and breathing. They are fairly far apart (60 beats/minute and 15 
breams/minute for humans) and is typically implemented in software as a digital filter. 

At block 410 the program computes the peaks in the high-pass filtered 
autocorrelation. The first peak will represent frequency of the motion of interest. Thus since 
the image rate f ( or frame rate) and the image number tp at the first peak is known, the 
frequency will be t^f Hz. Subsequent peaks may be used to obtain an average frequency. 
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^Referring now to figures 5;, 6 and 7, there is shown a graphical representation of the, 
subject invention as applied to an X-ray angiography sequence of a dog's heart. The frame 
rate fused is 15 frames per second. As can be seen, in figure 5 a plot is made of the 
correlation on the Y-axis versus the fame number represented on the x-axis. There is a strong 
correlation for approximately 60 frames, or .25 Hz (base frequency). There are also strong 
correlation for multiples of 60 frames - these are simply harmonics of the base frequency. In 
this example, the .25 Hz motion represents breathing motion (15 intakes per minute). 

An enlargement of the autocorrelation as shown on figure 6 shows an overlying 
periodicity in the range of the heartbeat (approximately frame 7, which corresponds to 2 Hz, 
or 120 beats per minute). 

A high-pass filter matched to the expected heart frequency range yields a filtered 
autocorrelation as shown in figure 7. The first peak in the high-pass filtered autocorrelation is 
at frame 8, corresponding to approximately 2Hz, or 120 heartbeats per minute, which is 
within the expected range of a dog's heart. 

Although the present embodiment has been described with respect to X-ray 
angiograms, the method of the present invention is not limited thereto. Furthermore, the 
method is not limited to Cardiology - breathing motion could also be detected with this 
technique in other applications. 

Periodicity is determined by using autocorrelation as described above, however other 
techniques may be used equally well, such as Fourier analysis. Also subsequent selection of 
the highest peak within a given frequency window may be used. 

For performance reasons, the method can be applied to subregions of the image. If the 
heart outline is known (as is the case in certain applications), then the calculations can be 
limited to this region. Alternatively, only every nth row and column (describing a grid over 
the image) can be used. 

The method can be applied to any given image sequence. For real-time applications, it 
may be more useful to apply it to a small retrospective window (1 to 2 seconds), and use it as 
a predictive measurement for the next 1 to 2 second window, on a continuous basis. 

Although the invention has been described with reference to certain specific 
embodiments, various modifications thereof will be apparent to those skilled in the art 
without departing from the spirit and scope of the invention as outlined in the claims 
appended hereto. 
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„ THE EMBODIMENTS QF THE INVENTION WHICH ANdEXCLUSIVE 
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS: 

1 . A method for measuring a periodicity of a recurring pattern contained in a time sequence 
of images comprising the steps of: 

(a) processing a plurality of said images for detecting a pattern ac'cording to a first 
criterion; and 

(b) processing the detected pattern according to a second criterion to determine a 
measure of periodicity of said recurring pattern. 

2. A method as defined in claim 1, each image in said time sequence being a two 
dimensional array of pixels. 

3. A method as denned in claim 2, said first criterion includes the steps of: applying an 
autocorrelation function over time t for each pixel; and computing an average 
autocorrelation for each image to produce an amplitude pattern of said average 
autocorrelation. 

4. A method as defined in claim 3, including the step of applying a filter to said amplitude 
pattern. 

5. A method as defined in claim 4, said second criterion including the steps of detecting a 
peak value in said amplitude pattern; and comparing said peak to its time of occurrence in 
said image sequence to thereby determine said measure of periodicity. 

6. A method for deterniining a periodicity measure from a sequence of images of a body 
having a plurality of motions therein, comprising the steps of: 

(a) processing a plurality of said images for detecting a pattern according to a first 
criterion; and 

(b) selecting from said detected pattern at least one of said motions according to a second 
criterion to determine a measure of periodicity of said selected motion. 
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7. A methodlfe defined in claim 6, said body includifig breathing an<Sieart motion and said 
selected motion being said heart motion. 

8. A method for determining a heart rate from a sequence of images of said heart, 
comprising the step of: 

(a) processing a plurality of said images to extract a periodicity measure therefrom. 

9. A method for determining a frequency of motion of an object from a time sequence of 
images of said object, said method comprising the steps of: 

(a) applying an autocorrelation function to individual pixels across said image 
sequence; 

(b) computing an average autocorrelation for each image to produce an amplitude 
pattern of said average autocorrelation; 

(c) applying a filter to said amplitude pattern; 

(d) applying a peak detector to said filtered amplitude; and 

(e) comparing said peak to its time of occurrence in said image sequence to thereby 
determine frequency of motion of said object. 
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